A Mixed-methods Exploration of the Outcomes of the Research-Aligned 


Mentorship program at Farmingdale State College 


Beverly Kahn,?* Kate Winter,° Erwin Cabrera,? and Lisa Cullington? 


@ Research Aligned Mentorship Program, Farmingdale State College (SUNY), Farmingdale, NY. 


> Kate Winter Evaluation, LLC, USA. 


RUNNING HEAD: FSC RAM Program Outcomes 


This work was supported by a First in the World FIPSE Grant awarded to Farmingdale State 
College by the United States Department of Education [P116F150028] from October 2015 
through September 2020. 


This paper was presented at the virtual Annual Conference of the Association for the Study of 
Higher Education (ASHE), November 18, 2020. It was selected for presentation through peer 
review of its abstract, conducted by ASHE members. 


FSC RAM Program Outcomes 


A Mixed-methods Exploration of the Outcomes of the Research-Aligned 
Mentorship program at Farmingdale State College 


Increasing 4-year graduation rates among US students, particularly those historically 
disadvantaged in higher education, is a national imperative. Using student data (N=1,462) from 
a federally-funded project, this paper examines the impacts of the Research Aligned 
Mentorship (RAM) Program on whether students are on track to graduate within four years. 
Qualitative and quantitative data and analytic approaches are integrated to explore significant 
outcomes, including year-to-year retention, annual grade-point average, annual credits earned, 
and whether students succeed in graduating at the end of four academic years. The RAM 
Program provides such interventions as a first-year experience seminar, collaborative learning 
workshops, intensive counseling, and mentored research experiences. Randomly-selecting 
treatment and control groups among low-income, first- generation, and minority students, the 
College achieved its key goal of a 20 percentage-point increase in the four-year graduation rate 
and achieved comparable gains in retention, GPA, and annual credits earned among students in 
the treatment group. Data show that active engagement, whether measured through 
participation in RAM interventions or by RAM staff perception, is positively and significantly 
associated with students having higher GPAs, annual credits earned, persistence from year-to- 
year, and 4-year graduation. 


Keywords: Undergraduate research, mentorship, student persistence, 4-year graduation, URM 
students, first generation college 


Introduction 

At its best, undergraduate education is transformative (Kahn, 2002; Scott, 2013). Through a 
variety of experiences that stretch from first year through senior year and occur both inside and 
outside the classroom, a qualitative change takes place. Such education broadens students’ 
perspectives, opens their eyes to new opportunities, and prompts serious reflection about self, 
values, and community (Pascarella & Terenzini, 2016). The transformative Research Aligned 
Mentorship (RAM) Program, assessed in this study, was established through a five-year FIPSE 
First in the World (FITW) grant awarded to Farmingdale State College (FSC) in 2015. The RAM 
program was intended to increase college success, as defined by completion of a 4-year degree 
within 4 years, as well as by milestone proxies (such as annual credits earned and year-to-year 
retention), among so-called “high needs” students who are frequently failed by mainstream 
higher education. A distinguishing feature of the methodology was the use of random-selection 
in choosing students for the RAM Program and a control group with matching characteristics. 


The purpose of this paper is to describe the RAM program: its underlying 
conceptual/theoretical framework, the components of the program, associated relevant 
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literature, the distinctive research methodology that was employed, the research findings, and 
the overall significance of this study. The research questions guiding this study are: 


1. What is the impact, both observed and perceived, of participation in the RAM 
program on annual retention, annual credits earned, GPA, and four-year graduation 
rate, as compared to non- participating students, controlling for student 
characteristics and prior academic achievement? 


2. Do impacts vary by level of participation, whether observed or perceived? 

3. Does participation in RAM activities increase student perceptions of sense of 
belonging in college or likelihood of pursuing research in careers or post-graduate 
study? 


The RAM Program 


Farmingdale State College (FSC) SUNY is a moderately selective four-year baccalaureate 
teaching institution that enrolls 10,000 students in its four schools of Arts and Sciences, 
Engineering Technology, Health Sciences, and Business. A disproportionate number of FSC 
students are “high need” (44% ethnic/racial minority, 59% low income, 44% first-generation 
college); 96% are commuter students who juggle family responsibilities and outside 
employment while pursuing a college degree. Aware of the multiple challenges and difficulties 
that beset high-need students and thwart their ability to graduate in four years, FSC sought to 
identify an effective model for success among high-need students: UCLA’s Program for 
Excellence in Education and Research in the Sciences (PEERS). In a 2015 study examining 12 
years of the PEERS Program, Brit Toven-Lindsey and colleagues demonstrated that PEERS 
interventions significantly improved the academic success of underrepresented minority 
students (Toven-Lindsey et al., 2015). Seven key interventions of PEERS are presented on the 
current website (Semel Institute for Neuroscience and Human Behavior, 2020). They are: (1) 
Pathways in Science seminars; (2) Collaborate learning workshops; (3) Career planning; (4) 
Research talks by UCLA faculty; (5) Social events; (6) Preparation for undergraduate research; 
and, (7) Research scholarships. 


While impressed with the PEERS example, FSC also recognized that there are differences 
between FSC and UCLA, as well as limitations to the PEERS model, that would need to be 
addressed: 


First, there are significant differences in the academic records and rates of persistence of 
students entering FSC in comparison to UCLA students (Table 1, Appendix 1). 


Second, given the UCLA students’ stronger records of prior academic achievement and 
graduation rates, the PEERS Program is a two-year program focused on launching high-need 
students well. FSC — with significantly lower 4-year and 6-year graduation rates - determined 
that its students would benefit from a carefully orchestrated four-year program designed both 


FSC RAM Program Outcomes 


to launch students in the first two years and then continue to provide guidance throughout a 
student’s college career. 


Third, PEERS students are drawn exclusively from those who major in the STEM fields minus 
Engineering. FSC wanted to cast its net broadly, serving students who pursue any of FSC’s 42 
baccalaureate majors offered by its four schools. 


Fourth, while UCLA recognizes the importance of offering workshops and tutoring in support of 
a student’s studies, FSC deemed it important to integrate both considerable academic and 
supplemental support components into its program. 


Fifth, while both UCLA and FSC see the benefit of having students engage in mentored 

research, PEERS does not actually match or place its students in research. Instead, given the 
resources of UCLA as a doctoral research university with 3,661 F/T faculty and 1,157 research 
staff, on-campus research opportunities are plentiful and PEERS students apply on their own for 
such positions. As a teaching institution with only 247 F/T faculty, FSC knew that if RAM 
students were to have research opportunities, the program staff would need to compensate for 
the fact that FSC lacks significant grant- funded researchers and major research labs. FSC would 
need to identify research opportunities both on campus and at other institutions, publicize 
those opportunities proactively, and then match RAM students with research placements and 
mentors. FSC also recognized that, given the heavy teaching commitment of its faculty, there 
would be a need for a financial incentive, support, and professional development so that FSC 
faculty might effectively mentor students in their research. 


Sixth, while UCLA students apply to participate in the PEERS program (submitting grades, Math 
SAT scores, and two essays), FSC sought to design a rigorous research study by employing a 
random control methodology that would see the annual selection of 400 entering high-need, 
first-year students (approximately 220 treatment and 180 control) by applying an algorithm to 
the database of all entering high-need students. Such anonymous random selection yields a 
cohort of treatment students representing a wide range of academic majors and varying 
records of prior academic achievement. While UCLA begins with high-achieving students and 
many colleges create special programs for honors students, FSC wished to pioneer a way to 
change the lives of all students. 


Conceptual Framework 

Inspired by PEERS, but also mindful of the differences between UCLA and FSC and in recognition 
of the limitations of the PEERS model, FSC created its Research Aligned Mentorship (RAM) 
Program using the following dynamic model (Figure 1). This conceptual framework depicts 
factors, relationships, and treatments that are designed to promote persistence, academic 
success, and four-year graduation rates among high-need students (e.g., low income, ethnic 
and racial minorities, and/or first-generation college students). 
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Figure 1. Conceptual Framework for FSC RAM 


efforts with active learning 


The FSC RAM conceptual framework begins by including the statistically confirmed and 
validated variables found in early models of student dropout (Bean, 1982; Tinto, 1975, 1987). 
Those variables are student goals and intentions, the student’s institutional experience of the 
academic system and the social system, and resulting academic and social integration. The RAM 
model then builds on these studies by incorporating a great deal of additional material that 
predicts student persistence and academic success: 


1. Astin’s (1984) student involvement theory in which the amount of student engagement 
in college (in both the academic environment and the social environment) is positively 
related to the amount of learning and personal development. 

2. Pascarella’s (1980) insight that, with regard to student-faculty interaction, the most 
positive influence comes from “interactions that extend the intellectual content of the 
study program into informal non-classroom contexts” (Aljohani, 2016, p.9). 

3. Insights on the importance of the first-year experience (e.g., Upcraft et al., 2015) . 

4. Institutional capacity-building can have a positive effect on student persistence through 
professional development of faculty and staff, the improvement of infrastructure (both 
physical 

5. and operating procedures/rules), the creation of digital communication modalities, and 
strengthening student support services. 

6. AAC&U findings about the connections between student success and high impact 
practices (e.g., learning communities, service learning, research with faculty, internship 
or field experience, project-based or active student-centered pedagogies, study abroad, 
and culminating senior experience). 
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7. More recent literature on the psycho-social needs of first-generation college students, 
low- income students, underrepresented minorities, and women in male-dominated 
fields, as well as related non-cognitive predictors of student success. 

8. Finally, most studies of undergraduate student retention examine attrition at 4-year 
residential institutions. FSC does not fit that traditional model, as 96% of FSC students 
are commuters who remain attached to their families and have home and work 
responsibilities. Accordingly, the RAM program puts a premium on learning how to 
engage commuter students and forge new bonds that create a vibrant on-campus family 
or, as it is called at Farmingdale, “RAMily.” 


Key Interventions/Components of FSC’s RAM Program 


First-Year Programming 

Successful transition from high school to college is vital for student persistence and success. 
The RAM Program strives to assure that its students not only adjust well to college, but also 
acquire a powerful identity as RAM Scholars and members of the “RAMily.” That effort begins 
in the spring with a congratulatory letter and glossy acceptance packet sent by the RAM 
Director to the 200+ entering students each year who are randomly selected for the program. 
Because they do not apply to participate in the program, because there is no scholarship 
awarded to participants, and because RAM Scholars will take on additional coursework and 
hands-on research, it is essential that the new students gain an early appreciation for the 
program, understand the importance of graduating in four years, and become enthusiastic 
about being RAM Scholars. They (and parents/guardians) personally visit the RAM offices in the 
summer to meet the Director and staff and to have individual advisement appointments with 
their first-year RAM Counselor. The new students attend a two-day RAM summer “Kick-Off” in 
which they meet the RAM staff, fellow members of their cohort, and upper-class RAM Scholars 
who will serve as their peer leaders throughout their first year. This mini-orientation — with 
information about college resources and break-out sessions featuring games and community- 
building — is effective in overcoming anxiety and in building RAM identity. Communication with 
peer leaders continues via social media over the summer. Then, an official “Welcome Event” for 
all new RAM Scholars and their family members is held during the first week of the semester. 
The College President congratulates the students and their parents, the Director and staff give 
brief introductions, current RAM Scholars speak about their research experiences and other 
RAM activities, and key faculty mentors are profiled. The program concludes with an official 
induction ceremony and reception. These three early events solidify the new RAM cohort. 
Annually, no more than 4 of the 200+ students elect not to participate in the RAM Program. 


Once the fall semester begins, all new RAM Scholars take RAM 101 (First Year Seminar) that is 
facilitated by their RAM counselors. Meeting weekly in small classes of 12-20 students, this 1- 
credit course features group work, active listening, and class discussions that prepare students 
to navigate college. They are guided through opportunities to reflect about themselves, each 
other, their academic pathway, and their career and personal goals. They journal weekly in a 
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“Digital Roadmap to Graduation and Beyond” and view and discuss TED talks such as Angela 
Duckworth’s “GRIT” (2013), Daniel Goldstein on “The Battle Between your Present and Future 
Self” (2011), and Sheryl Sandberg on “Women Leaders” (2010). This weekly collaborative 
gathering builds support and friendship among students and bonds them with their counselor. 


Finally, all first-year students are encouraged to join in social and co-curricular programs 
offered to all RAM Scholars throughout the year. These include holiday parties, ethnic 
celebrations, community service, roller skating, and assorted speakers on topics such a Big 
Data, Health Disparities in NYC, and Biomedical Research at NYU. 


Intensive Counselling Over Four Years 

Given the goal of seeing RAM Scholars graduate in four years and the current weak graduation 
rates at FSC (4 yr: 31% and 6 yr: 53%), the RAM program instituted an intensive-type of 
counseling that includes much more than academic advisement. While students are expected 
to see their department chair or a faculty advisor in their major once each semester, they are 
also required to meet with their RAM Counselor at least two times each semester during years 
1 and 2 and at least once per semester during years 3 and 4. This program is successful in 
guiding and motivating students, helping students navigate the complexities of higher 
education and address the weak sense of belonging and lower self-efficacy that is often 
associated with high-need students. They are guided along the Digital Roadmap’s path of self- 
reflection and engage in short- and long-term planning. The bond created between the 
counselor and advisee in the fall is strong as students are assigned a counselor according to 
their major, and that counselor teaches the advisee weekly in the RAM 101 seminar. As a result, 
first-year students actually have one-on-one appointments with their RAM counselor not the 2 
times that are required, but on average 6 times a semester. Once the habit of meeting with the 
counselor has been established during the first year, it continues through years 2, 3, and 4. 


During their second year the students transition to their new counselor, either the Director or 
the Assistant Director of the program. The counselors’ new focus is preparing their advisees for 
their research experiences and helping them with post-graduation plans. Majors in STEM or 
health professions work with the Director (who holds a PhD in Biology) and the other students 
with the Assistant Director (who has a social science background and PhD in Education). In 
addition to meeting for advisement, the Director and Assistant Director see all of their advisees 
weekly during the sophomore year as they are the instructors of the 1-credit RAM 201 
(Introduction to Research). As mentioned previously, having those who counsel the students 
also teach RAM courses serves to solidify the bonds between students and the staff who are 
there to help them succeed. 


Academic Programming — Curricular Enhancements and Active Student-Centered 
Pedagogies 

The RAM program is not only a support program. RAM is an academic program in which 
students enroll in RAM-specific courses that have been approved by the College as satisfying 
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major or other graduation requirements. RAM 101 (First Year Seminar) and RAM 201 
(Introduction to Research) are both credit- bearing courses. Another important RAM course is 
RAM 102, a 1-credit Collaborative Learning Workshop facilitated by the math professors who 
teach RAM-only sections of statistics and pre-calculus. The collaborative learning workshop is 
based on the pioneering work of Uri Treisman (1992) who understood that disadvantaged 
students are able to prosper when collaborative workshops assist them in foundational math 
courses. Internal assessment shows that RAM Scholars earn grades in their companion math 
course that are statistically higher than those of control group students who do not benefit 
from RAM 102. RAM 303 is a 3-credit research experience in which a RAM Scholar engages in 
research with an FSC faculty mentor who determines the grade for RAM 303. Finally, beyond 
the specific courses that carry the RAM designation, the program also arranges for sections of 
engaged pedagogy courses reserved exclusively of RAM students. These courses include a small 
(i.e., 25 students) integrated Introduction to Biology lecture and lab for Biology majors, 
Learning Communities in which students enroll two courses that faculty synchronize around a 
common theme, and other project-based learning courses from across the curriculum. 
Wherever possible, RAM Counselors assure that RAM Scholars choose to register for course 
that employ high impact pedagogical practices. 


Research Preparation and Practice 

The culminating experience of the RAM Program is the placement of each RAM Scholar ina 
research experience on- or off-campus. This placement gives the Scholar a high-impact hands- 
on learning experience. Whether students participate in an NSF Research Experience for 
Undergraduates, apprentice in a business internship, or work side-by-side on campus with a 
faculty research mentor, this educational experience outside of the normal classroom enhances 
knowledge and provides scholars with direct exposure to real-world applications and career 
options. See Appendices for sample testimonials from student researchers. 


The RAM Directors work diligently to secure placements by conducting one-on-one interviews 
with over 150 FSC faculty researchers and by building relationships with researchers at other 
universities and laboratories and with businesses. Each option is profiled in a database available 
to students. 


Unlike most university programs that simply thrust students into the research setting without 
serious preparation or training, all RAM Scholars enroll in RAM 201 (Introduction to Research) 
in their sophomore year. Taught by the RAM Directors, this 1-credit course covers how to find 
and read research articles and how to break down the complex ideas of a research proposal. It 
walks students through the scientific process and issues of research ethics. All students are 
required to complete an IRB-compliant training in research with human subjects. They conclude 
the course by creating a research proposal in digital form. Ancillary workshops are offered on 
how to apply for NSF-REUs and other competitively funded programs. 


FSC RAM Program Outcomes 


Once a Scholar has completed RAM 201, the RAM Directors then meet with the student to 
select and secure the best summer or academic year placement. Because the Directors have 
come to know the students in RAM 201 and as advisees, they are able to take into account not 
only the students’ academic and career interests, but also their personality and mentoring 
style. 


Most research placements carry the RAM 303 (Research Experience, 3 cr.) designation. 
However, students have the option of earning 6, 9, or 12 credits depending on the hours 
worked. The RAM Guidebook that is shared with students, faculty mentors, and off-campus 
supervisors and mentors requires that the student complete a detailed agreement with the 
mentor, maintain a log of hours and activities, prepare weekly reflections on the learning 
experience, and respond to various prompts about their experience as part of mid-term and 
final evaluations. The research mentor also prepares evaluations and sustains ongoing 
communication with RAM Directors. Students are encouraged to join their research mentors in 
proposing to submit a poster or paper at professional conferences. 


Creating the RAMily by Engaging Students in Extra-curricular and Co-curricular Activities 
Because almost all RAM Scholars are commuter students who are tied to family, the RAM 
Program strives to nurture in its students the sense of community and engagement in college 
that students at residential institutions come by naturally. Not only are RAM Scholars 
encouraged to join FSC clubs, organizations, and student government, but the RAM staff work 
diligently to create family (or “RAMily”) among all 800+ RAM Scholars. It begins during the 
summer Kick Off orientation when students play a raucous game of “RAMily Feud.” It continues 
with speakers, social events, workshops, film nights, and parties throughout the year. An 
underlying theme of many RAM activities is Social Justice. To that end, two years ago “RAMdom 
Acts of Service” was launched where students are given monthly opportunities for community 
engagement and service (e.g., beach and highway clean up, volunteering for Cancer Walks, 
serving food at homeless shelters). A really powerful service experience is the Alternative 
Spring Break program where 25 students spend their vacation out-of-state working for Habitat 
for Humanity. 


Lasting friendships and bonds between students and with RAM staff are forged by such an 
intense experience away from home and family. Engagement and RAMily are important 
ingredients that contribute to student persistence. 


Infrastructure 
For a program to serve students effectively, supportive infrastructure (often referred to as 
scaffolding) is essential. Key components of the RAM infrastructure are: 


(1) Physical Space. The RAM Program is centrally-located in the College Library, an 
attractive building that also houses the tutoring center, writing center, and faculty 
development center. The RAM suite of offices includes a large conference room that 
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is used by students throughout the day for study and socializing. Some call it RAM 
CENTRAL. 

(2) Priority Registration. The overriding RAM goal is to see students graduate in 4 years. 
At many institutions a barrier to graduation on-time is the fact that students are 
thwarted from registering for required courses in the normal sequence because they 
are closed by the time the student registers. RAM Scholars avoid that stumbling 
block because the Registrar accords them the privilege of priority (e.g., first day) 
registration for courses during all four years at FSC. 

(3) Faculty Development. Faculty are the mainstay of college education. The RAM 
Program relies on faculty members to mentor students in research effectively, to 
engage students with active learning pedagogies, and to practice culturally- 
responsive teaching and promote social belonging for their high-need RAM Scholars. 
To that end, RAM has worked to assure that faculty are provided with professional 
development opportunities and programs that will help them to become better 
teachers, mentors, and allies of RAM Scholars. 


Literature Review 

The RAM Program is informed by earlier research and studies that address: (1) student 
retention, (2) the needs and challenges of high-need students, and (3) interventions and 
remedies that appear to be effective in advancing academic achievement and 4-year 
graduation rates among high-need students. Classic studies of student retention (e.g., Tinto, 
Bean, Astin, Pascarella) were cited earlier in this paper with reference to the dynamic model on 
which the RAM Program is grounded. A considerable literature base exists regarding 
undergraduate student success in college, whether specific to student characteristics such as 
first generation college or low income (see for example, Thayer, 2000), or within specific fields, 
such as science, technology, engineering, and mathematics (STEM) (Tsui, 2007). This review 
provides a brief overview of the relevant literature behind the development, implementation, 
and assessment of the RAM program. 


As noted, the RAM program was intended to increase the proportion of college students 
graduating within four years. One “High Impact Practice” (Kuh, 2008) selected for RAM was the 
First Year Experience (FYE) course. Use of these courses to familiarize students with the 
expectations of college and to introduce them to a cohort of peers has been well documented, 
particularly with high needs students or students in developmental courses (What Works 
Clearinghouse, 2016). First-year experience courses have been found to have a pronounced 
impact on the future trajectory of students (Fitzpatrick et al., 2020; Harvey et al., 2006; Upcraft 
et al., 2015). Interactions with the cohort and the RAM staff offering the FYE course were 
intended to foster increased sense of belonging in college, and at Farmingdale specifically, 
which has been shown to be important to college student success, especially for traditionally 
underrepresented groups such as the high needs students eligible for the RAM program 
(Bowman & Brandenberger, 2012; Hurtado & Guillermo-Wann, 2013; Maramba & Museus, 
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2011; Museus et al., 2017; Walton & Cohen, 2007) and within STEM fields (Estrada-Hollenbeck 
et al., 2011; Hurtado, 2007). 


Regarding high-need students, challenges faced by first-generation college students include 
issues related to personal adjustment, cultural fit, financial stress, weak self-confidence 
regarding academic preparation and academic abilities, and lower self-esteem (Aspelmeier et 
al., 2012; Bettencourt et al., 2020; Chen, 2005; Vuong et al., 2010). Underrepresented minority 
students face similar challenges (Hernandez et al., 2013; Walton & Cohen, 2011). Researchers 
have examined psycho-social needs of high-need students such as the dealing with the 
“imposter syndrome” (Krumrei-Mancuso et al., 2013; Reynolds & Weigand, 2010; Sparkman et 
al., 2012). The RAM Program implements multiple interventions that stretch across all four 
years of the student’s college experience to build community, enhance sense of belonging, 
combat feelings of isolation and of being an imposter, and help students access needed 
resources to be successful. 


The RAM Program practices and encourages engaged pedagogies and high-impact practices 
such as learning communities, collaborative learning workshops, enhanced gateway courses 
that might include supplemental instruction, service learning, and culturally-responsive 
strategies for teaching and relating to high-need students (Duncan & Dick, 2000; Koch et al., 
2017, 2018; Kuh, 2008; Mack et al., 2019; Peterfreund et al., 2008; Rocconi, 2011; Treisman, 
1992; Webb, 1989). The RAM program also developed Collaborative Learning Workshops, 
which have been shown to encourage problem-solving and teamwork (Toven-Lindsey et al., 
2015; Treisman, 1992), as well as student engagement, which is well connected to student 
persistence in college (Astin, 1984, 1985). Studies have also identified benefits of connecting 
FYE courses with mentoring (Fitzpatrick et al., 2020), which RAM did. RAM students engaged in 
mentoring from the RAM faculty and staff, as well as from their peers in the RAM program. 
Peer mentoring has been shown to be beneficial to student persistence (Mavrinac, 2005; 
Strayhorn, 2014; Waitoller & Artiles, 2013). Beyond mentoring, the RAM program built in 
intrusive advising from RAM-specific counselors, because such holistic advising and coaching 
are associated with enhanced student outcomes (Bettinger & Baker, 2011; Lee et al., 2009; 
Rodgers et al., 2014). 


Finally, as the name suggests, the signature feature of the Research Aligned Mentorship 
Program is mentored research. It is a powerful intervention that RAM Scholars experience. 
There is a wealth of resources on mentored research (Byars-Winston et al., 2015; Hensel, 2018; 
Ishiyama, 2007; Lopatto, 2016; Pfund C, 2016; Prunuske et al., 2013; Temple L, 2010). While 
many studies of the benefits of early college research experiences focus on STEM fields (James 
& Singer, 2016; NASEM, 2016; PCAST, 2012; Rosenberg et al., 2017; Saw, 2020; Tsui, 2007), 
these expereinces have been shown to benefit students outside of the STEM fields, as well 
(Hurtado & Guillermo-Wann, 2013; Sell et al., 2018; Walters, 2018). RAM actively worked to 
create and/or identify opportunities for undergraduate student research both on- and off- 
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campus, in part by working with faculty to develop capacity to conduct research with students 
and not through course-based opportunities such as CUREs (Russell et al., 2015). 


Positionality 

This research is based on a federally-funded project on which the first author was principal 
investigator, the second author was the external evaluator, and the third and fourth were 
project directors. Beverly Kahn has had a 47-year career in academia as a Professor of Political 
Science and then College Administrator (Dean of Arts and Sciences, Associate Provost, VP for 
Internationalization, and Provost/Academic VP) at five institutions. During that time, she 
secured funding for - and implemented — a great number of major student-focused projects 
(FIPSE, Title Ill, SSS-TRIO, NSF S-STEM). This background equipped her with the knowledge, 
skills, experiences, and values that she was able to draw upon when envisioning, creating, 
implementing, and assessing the RAM Program. Kate Winter has been conducting evaluations 
of federally-funded programs since 2003, specializing in projects to broaden college 
participation (particularly in science, technology, engineering, and mathematics). As Director of 
the RAM Program, Erwin Cabrera is able to relate to the RAM Scholars because he shares their 
background. He is an underrepresented minority, first-generation college student and child of 
immigrants who excelled as an undergraduate at the University of Maryland, Baltimore County 
where he was a Meyerhoff Scholar and participated in the NIH Health Minority Access to 
Research Careers Program. He earned a PhD in Biomedical Sciences from the NYU School of 
Medicine and, in 2016, joined the FSC RAM Program, ready to give back, providing students like 
himself with an enriching college experience. Lisa Cullington is the Assistant Director of the 
RAM Program who interacts with RAM Scholars daily as an adviser and instructor of RAM 201 
and who also matches many students with research mentors. She relates well with high-need 
students as she grew up in a low-income household and was a first-generation college student. 
Before coming to FSC, she spent several years as a K-12 educator of immigrant students and 
earned a PhD in Urban Education, Leadership, and Policy Studies from the University of 
Massachusetts. 


Methods 


This study used mixed methods to collect and interpret qualitative and quantitative data from 
multiple sources in order to understand both the impact of the RAM program on participants, 
as well as the efficacy of the program model. While the funding agency prioritizes quantitative 
assessment of observed academic outcomes (i.e., enrollment, retention, grade-point average, 
graduation), this study incorporates student perceptions of experiences, as well as the 
perceptions of project staff, to obtain a more complete picture of lived experiences within the 
RAM program. An integration of methods (Greene, 2008) took place at the evaluation design, 
data collection, and analysis levels. Real-time integration of qualitative and quantitative 
methods is well supported in evaluative research (Mason et al., 2020; Perfetti et al., 2020). 
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Data and Variables 

Institutional records provided data on year-to-year retention, annual GPA, annual credits 
earned, and graduation status, as well as student characteristics such as gender, race/ethnicity, 
first-generation college status, prior academic achievement, and Pell eligibility status. Program 
records provided details of random assignment to the treatment or control condition, as well as 
compliance with assignment and participation in interventions. A pre-post survey instrument 
administered during the first couple weeks of entering college and again during the month of 
graduation solicited student perceptions of aspects of college adjustment, sense of belonging, 
sense of community, sense of self-efficacy in completing college, and involvement in campus 
activities. Additionally, RAM program staff used a rubric to score levels of student engagement 
with the program, which allowed comparison with attendance data. Student-level data was 
matched using a deidentified study code to protect student confidentiality. 


In general, completing a year of college and returning the next year are highly correlated. For 
this reason, the quantitative outcomes explored included a progression over time regarding 
whether a student is “on track” to graduate within four years; specifically, whether students 
end the first year with at least 30 credits, the second year with at least 60 credits, et cetera. 
Some students stop-out for single terms during the year or do not maintain full-time credit 
loads. Monitoring the earned credit accumulation at the end of each academic year permits a 
general understanding of the likelihood of students finishing a 4-year degree in four years. 
Additionally, average annual GPA was explored as a proxy for academic success and likelihood 
of graduating in four years. In addition to the student characteristics and baseline score of 
academic achievement, whether students entered through an associate degree ora 
baccalaureate degree program is included as a covariate, as it relates to credits earned and 
progress to degree. Figure 2 provides an overview of the study design, showing the various 
types of data across the stages of the program. 
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Qualitative Quantitative 


RCT: Random Assignment 


to Study Conditions 
Pre Student Surveys: 
All entering RAM and 
Control-assigned Students 
a Invited 
Program 

Start Open-ended responses about feelings, Scaled response items about sense of 
perceptions, experiences belonging/community, self-efficacy to 


finish college, campus engagement 


Treatment Student Focus Groups & Phone 
Interviews with Program Staff and Faculty: Early 
in Implementation of the Program for Formative 


Feedback on Activities 
, Observations ‘Ns 
RAMily scores of perceived level of engagement, Student participation in RAM activities, 
Tiecuhicalt scored by core program staff collected through course enrollment and 
attendance records 
Program 


Official student records for enrollment, 
credits earned, grade-point average, and 
graduation 


Post Student Surveys: 
All graduating RAM and 


Control-assigned Students 
At rs Invited “a 


Graduation 
Open-ended responses about feelings, Scaled response items about sense of 
perceptions, experiences belonging/community, self-efficacy to 
finish college, campus engagement 
Analytic Open and axial coding Descriptive and inferential statistics 
Approach 


Figure 2. Mixed Methods Study Design 


Since Fall 2016, there have been four cohorts of students in the RAM program study. Across the 
cohorts, there are 1,462 students, with 839 students assigned to the RAM treatment group and 
623 non- RAM control group students. Baseline equivalence was not established for the 
treatment and control groups on one of the What Works Clearinghouse (WWC, 2019) baseline 
covariates of interest for postsecondary education research: prior academic achievement 
(scored on the SAT 2016 scale), with RAM students scoring slightly higher (M=911) than control 
students (M=874; t=4.11; df=1,182; p<.001). The standardized effect size of the difference is 
very small (Cohen’s d = 0.2). There are no statistically significant differences between mean 
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scores of the treatment or control groups on the other WWC covariate, Pell eligibility, or by 
gender, URM status, or first-generation college status. All covariates are included in the 
regression models to account for potential selection bias. Only students with data for all 
variables are included in the quantitative analysis. 


While all cohorts have completed the pretest, only cohort 1 has completed the posttest. 
Further, the posttest survey was administered to graduating cohort 1 students during the peak 
of the COVID-19 global pandemic when all activities with students were forced online, and the 
response rates were far lower than desired (21% overall response rate; 28 of 101 RAM students 
(28%) and 6 of the 61 (10%) control students). Similarly, with the majority of the second 
academic term of the 2019-20 year taking place virtually, it was deemed inappropriate to 
attempt to have program staff determine RAMily scores for this academic year, which are 
calculated by program staff after they spend the first two years with students. The pandemic 
impacted the second year for cohort 3 and the first year for cohort 4, resulting in study scores 
being unavailable for many students. 


Only cohort 1 has been in the study long enough to have returned for and completed their 
fourth year, while cohort 2 has outcome data through returning for and completing their third 
year, cohort 3 has outcome data through returning for and completing their second year, and 
cohort 4 has outcome data through completing their first year. Accordingly, the quantitative 
analytic sample sizes vary by outcome measure, with the largest samples for the outcome 
variables in earlier years. Similarly, staff observation data are available for only the first two 
cohorts and pre-post survey responses are only available for cohort 1. An overview of the 
progression of each cohort over time through the 4-year program by treatment or control 
condition is provided in Table 2 (Appendix 2). 


Only one of the control-assigned students engaged in any of the treatment interventions and 

19 (2.3%) of the treatment-assigned students did not engage in any activities (Table 3, Appendix 
3). The largest portion of treatment-assigned students (330 of 839; 39.3%) engaged in 4 of 6 
interventions. As cohort 4 would not have yet engaged in their year 2 activities, and not all 
cohort 3 members had finished their research due to summer cancellations due to COVID-19, 
these figures are more meaningful by cohort, as provided in Table 4 (Appendix 4). As only the 
one control-assigned student engaged in activities, only RAM students are depicted in Table 5. 


Table 4 provides the portion of RAM students who engaged in each intervention. Since not all 
RAM students needed to take the mathematics courses to which the collaborative learning 
workshops were attached, this intervention was not required of all RAM-assigned students. 
While the mentored research experience is a cornerstone of the RAM program, not all students 
complete the experience as early as intended in the program (i.e., in the summer following 
their second year). Summer 2020 saw several students’ planned research cancelled due to 
COVID-19 travel and gathering restrictions. The College was closed to in-person teaching and 
research and many off-campus research placements were cancelled. While cohort 4 was 
technically engaged in their year 1 activities and not engaged in the second-year experience 
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(Introduction to Research Course) or mentored research in year 1, several students participated 
early. 


RAM activities that begin in the first year (i.e., the FYE course, RAM counseling, and the use of 
the digital “roadmap to 4-year graduation”) were experienced by a majority of each cohort 
(95% - 100%). Participation corelates too highly between these three variables to explore them 
individually (r=.97; r=.97; r=.99; all p<.01), which makes it impossible to tease out quantitatively 
any unique influence each intervention may have had. Supplemental analysis drawing on 
qualitative data from participants will be conducted to understand perceived relative 
contributions to their success. In the meantime, an overall “dosage” score was calculated to 
permit exploration of a potential cumulative impact of all of the interventions (distributions 
provided in Table 3). 


Analysis 

While this intervention was conducted through a Randomized Controlled Trial (RCT), students 
at colleges and universities do not necessarily adhere to their randomly-assigned conditions, 
which can make detecting true treatment effects challenging. To better understand the actual 
treatment effect on students who engaged in programming, as well as to explore potential 
differential impacts of exposure to different interventions and numbers of interventions, we 
also explored “treatment on the treated,” controlling for baseline covariates to account for 
selection bias. This paper provides the findings from this analysis. Quantitative analysis includes 
descriptive statistics, independent samples t-tests, bivariate correlations, and multivariate 
regression (logistic regression for the dichotomous outcomes of “on track” and “graduated”). 
All tests of significance use an alpha of .05. Quantitative findings were explored in the context 
of qualitative data from matched students, when available, using both RAMily scores and 
survey responses. Qualitative data were open and axial coded to identify core elements in 
students’ perceptions and shared experiences, as well as themes across students. 


Findings 

Institutional Data 

Per the conceptual framework (Figure 1), engagement in treatment activities was intended to 
increase participants’ college self-efficacy, sense of belonging, annual GPA, number of earned 
credits, and year- to-year retention, which were all predicted to increase 4-year graduation 
rates. Using institutional data and the measure of student engagement in treatment activities 
(dosage), all but the first two outcomes could be tested. Monitoring over time, there are 17 
outcome variables explored, representing annual GPA, credits earned, whether students were 
“on track” with their cumulative earned credits to graduate in four years, whether they 
returned the following year, and measures of cumulative GPA during the study and whether 
they graduated. 


For each outcome, separate regression analyses were conducted, controlling for whether the 
student is female URM, male URM, male white (all compared against female white), Pell 
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eligible, academic achievement baseline (bottom and top thirds as compared to the middle 
third), first-generation college, and whether the student entered through an AAS program, 
rather than a BS program (which is not applicable for year 4 outcomes, as no cohort 1 students 
were AAS). As noted, sample sizes vary by outcome as not all cohorts have completed all four 
years. Further, only the measures of “returned” and “graduated” are explored against the full 
entering sample; other analyses follow only persisting students for the treatment and control 
groups equally. Because the focus is on whether and to what extent the treatment influences 
the outcomes controlling for other factors, only the coefficients for “dosage” on each of the 17 
individual outcome variables are provided with the sample size and significance level (Table 5, 
Appendix 5). 


As can be seen in Table 5, higher scores on “dosage” positively associate with all of the 
intended outcomes, except for the number of credits earned and GPA in the fourth year. 
Students who actively participated in RAM treatments were more likely than students who did 
not participate to perform better in their first three years in their coursework (annual GPA), to 
earn more credits each year, to return the subsequent year, and to graduate within four years. 
While the coefficients are small, they demonstrate about a 20% increase in the likelihood of 
being “on track” to graduate within four years for each of the first three years of participation. 
Further, students with higher engagement in treatment activities are 14% more likely to 
graduate within four years. This provides evidence that the RAM interventions were successful 
in increasing 4-year graduation rates of participating students. 


RAM Staff Observations 

RAM staff provided engagement ratings for all RAM students, which were combined into an 
overall “RAMily” score. These scores were based on the level of familiarity each staff member 
had with each student and their perceptions of student engagement with the program. As a 
largely commuter campus, it is not uncommon for students to be on campus only long enough 
to attend required classes and activities, without spending time in community with faculty, 
staff, or other students (Burlison, 2015). The RAM program intended to support students in 
making time for community-building and engagement. It was predicted that the RAMily scores 
would correlate more highly with dosage scores, but it appears that a sizable portion of RAM 
students were able to participate in high numbers of RAM interventions without spending 
enough time with RAM staff to score highly in their ratings (r = .47; p<.01; Table 6, Appendix 6). 
RAMily scores have higher correlations with being “on track” in years 3 and 4 and with 
graduation, than do dosage scores, but dosage correlates more highly in the first two years. 
This means students who continue to engage closely with the RAM program after the first two 
years are more likely to do better in their final two years and, ultimately, to graduate on time. 
RAMily also has a higher correlation with cumulative GPA than does dosage. As predicted, being 
scores for whether a student is “on track” to graduate within four years correlates highly with 
whether students actually graduate in four years. 
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Student Perceptions 

With only 34 graduating students completing the posttest, this subsample of data does not 
have much statistical power. Responses were matched to the quantitative data set and 
explored for relationships with RAMily scores, dosage, and whether students were “on track” to 
graduate each year. Only four items had statistically significant correlations. Responses to the 
item, “lam satisfied with the extent to which | participated in social activities,” correlates with 
RAMily (r=.47, p<.01), dosage (r=.42; p<.05), and being “on track” at the end of both year 2 
(r=.40; p<.05) and year 4 (r=.58; p<.01). The item, “I feel like a part of Farmingdale State 
College,” correlates with RAMily (r=.42; p<.05), dosage (r=.35; p<.05), and being “on track” at 
the end of year four (r=.47; p<.05). Likewise, responses to the item, “I have several close ties at 
this college,” correlate with dosage (r=.38; p<.05) and with being “on track” at the end of year 
four (r=.52; p<.01). Finally, responses to the item, “Are you considering a major or career in 
research,” correlates at a statistically significant level only with dosage (r=.42; p<.05) and 
RAMily (r=.40; p<.05). These connections provide some proof of concept that participating in 
RAM activities associates with increased perceptions of sense of belonging, as well as with 
increased likelihood of pursuing majors or careers in research. 


Students were also asked to provide “in their own words” their plans for after graduation. 
While most students offered details about graduate programs to which they had been 
accepted, or jobs they had already secured in their fields, one offered a specific reflection on 
the impact of participating in the RAM program: 


After graduation | plan to work in my field in a full time position. | am actually grateful 
now to have a starting job in the field, even though it might not be the perfect job for 
me, it is good experience. | would have never been able to be in the position that |am in 
without the support of Farmingdale State College and the RAM program. 

— RAM 2020 Graduate; Dosage score 4 (of 6); RAMily score 7 (of 25) 


That this student chose to credit the school and RAM program is particularly interesting given 
that this student was not scored highly by the RAM staff and had an average dosage score. 
Additional items are being added to the posttest in order to solicit additional reflections on the 
impacts of the RAM program by activity to increase the ability to understand how the program 
positively impacts students’ persistence and graduation. 


Distinctive Achievements — Research Placements 

The superior performance of RAM Scholars is reflected in their rates of retention, cumulative 
GPA, and mean credits earned year-to-year. In addition, a truly distinctive achievement of the 
RAM Program is the participation of RAM Scholars in mentored research experiences outside of 
the normal classroom. Prior to the creation of the RAM Program, only a handful of FSC students 
had engaged in research with faculty on campus and about 2-4 a year participated in off- 
campus placements. That changed with the arrival of the RAM Program. To date, from fall 2017 
through summer 2020, 301 RAM Scholars have participated in research — 181 in on-campus 
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placements with faculty and 120 in off-campus placements at other universities, research labs, 
and businesses. Additionally, for the first time ever, FSC students (all RAM Scholars) applied for 
and won competitive scholarships for summer research. The funding sources included: NSF 
REU’s, the Leadership Alliance, the NYU Undergraduate Intern Program, and the Summer 
Health Professions Education Program. It should be noted that, for summer 2020, nine more 


RAM Scholars were offered funded off-campus research; unfortunately, those programs were 


cancelled due to the COVID-19 Pandemic. A partial list of distinctive off-campus placements 
includes: 


Bowie State University 
Broadhollow BioScience Park 
Brookhaven National Labs 
California State University Chico 
CUNY Building Performance Labs 
Feinstein Institute 

John Jay College - CJ 

Johns Hopkins University 
Memorial Sloan Kettering Med Center 
Montana State 

Morgan Stanley 

Rutgers University 

Stanford University 

Stony Brook University 
University of California Berkeley 
University of Florida 

University of Miami 

University of Nairobi Kenya 
University of Nebraska 


Discussion, Lessons Learned, and Implications 

This study establishes that minority, low-income, and first-generation college students - who 
disproportionately fail to complete college - thrive and succeed when they experience a 
carefully- orchestrated four-year program that engages them in curricular, co-curricular, and 
extra-curricular programs and activities. The more opportunities for engagement that are 
provided across all four undergraduate years, the greater the impact. The following list 
identifies key lessons that are derived from the success of the RAM Program at Farmingdale 
State College: 


(1) 


With a carefully-orchestrated program, any institution (and especially those like FSC 


that are second- tier public institutions where the bulk of America’s high-need 
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(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


students attend) can change the lives and trajectories of their students, including 
those who arrive with lower levels of prior academic achievement. 

The first two years are key. RAM orientation and welcome programs, as well as a 
First Year Seminar for RAM students, ease adjustment to college. Collaborative 
Learning Workshops attached to problematic gateway courses (such as in 
mathematics) enable students to learn and perform better, while active student- 
centered pedagogies such as learning communities in the first years engage and 
motivate students. 

Holistic advisement and counseling across all four years are important supports and 
guides for students. Due to their close relationship with their advisees, RAM 
counselors are able to identify problems, help the students overcome them, and 
thus help students complete their graduation requirements. 

Building a close-knit community among students that fosters belonging and 
engagement contributes to student satisfaction and persistence. Through RAM 
coursework, service-oriented projects, social activities, field trips, peer mentorship, 
and ongoing social media communication, RAM Scholars engage with each other and 
with RAM staff. The result is a strong bond and support system among students — 
which is especially important because 96% of RAM Scholars are commuter students. 
Creating “RAMily” is hard work, but it clearly pays off with superior year-to-year 
retention of RAM Scholars. 

Embedding a program in the academic curriculum is important. RAM is more than a 
support program. It provides students with credit-bearing RAM and RAM-approved 
coursework that counts toward graduation requirements. Students appreciate that 
RAM courses are central — not ancillary — to their degree and they understand that, 
by accumulating credits they are making progress toward graduation. 

Successful and rigorous programs for high-need students that contain research 
components need not be restricted to honors students. RAM is not an honors 
program. FSC’s anonymous random selection of students yields a cohort of students 
with varying records of prior academic achievement. Yet, through the carefully 
orchestrated and multifaceted program, combined with peer support and 
community, RAM scholars succeed, complete serious research projects, and 
graduate. 

Institutions can implement effective programs without awarding scholarships to 
students. This lesson is especially important for institutions like FSC without large 
endowments. 

Experiential Learning through mentored hands-on research is powerful and often 
transformative. The RAM experience also teaches that mentored research is most 
effective when students are first given a formal introduction to research before they 
are thrust into a strange and intimidating research setting. Furthermore, the 
Farmingdale success in placing students in research on campus with faculty mentors 
and also off-campus in research at other universities and labs (including helping 
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them apply for competitive summer grants such as NSF REUs) indicates that any 
institution can make a commitment to student research. Even at primarily teaching 
institutions that do not have graduate programs and extensive research labs, faculty 
can give their students mentored research experiences that enhance learning and 
open new doors for post-graduate study and careers. 

(9) Early and continued buy-in from the college community is important for the success 
and institutionalization of a new program. Leadership is key. The RAM program was 
led by the former Provost of the College. Even while designing the program, she 
consulted with all constituents — the President, Provost, Deans, and Department 
Chairs, as well as key faculty and professional staff. Once the program was launched, 
she assured that faculty were supported and became engaged and that program 
successes were communicated widely across the College. An internal campaign to 
highlight a program’s achievements is important. Ongoing recruitment, training, and 
involvement of more faculty each year (as the program grows and new faculty are 
hired) creates a bulwark of enthusiastic support that, in turn, cements the 
institutionalization of a new program. 


Conclusion 

The RAM Program at Farmingdale State College in New York was born as an answer to a difficult 
question: What can be done to raise the 4-year graduation rate of students who are under- 
represented minorities, first-generation college, and low income? Based on a model from UCLA 
— an institution quite removed from the reality of a second-tier commuter campus like FSC — 
and cognizant of what the vast literature on student attrition versus persistence considers 
important, a successful grant was written that allowed FSC to serve as an education laboratory. 
As part of the experiment, it was hypothesized that, if one created a multifaceted program, the 
4-year graduation rate could climb by 20%. percentage-points. As good scientists, a rigorous 
evaluation of the results was proposed: a randomized selection of a treatment group and a 
control group, along with a battery of quantitative and qualitative measures. In the end, the 
experiment succeeded. Something new was created that raised the 4-year graduation rate of 
the treatment group by 29% over the 2015 IPED’s rate that was cited in the FIPSE grant 
application (and 18% over the 2018 IPED’s rate). 


It needs to be acknowledged that an unexpected variable did bedevil the experiment. A global 
pandemic did contaminate some of the results. The first - and most important - cohort of 
students who matriculated in fall 2016 was in its very last semester before graduation in March 
2020 when the virus caused the College to end in-person classes and other on-campus 
activities; the College shifted to remote learning. Many components of the RAM Program were 
affected — no in-person RAM counselling, no Alternative Spring Break with Habitat for 
Humanity, no RAMdom Acts of Service, no events in the RAM Conference Room (aka RAM 
Central), no on-campus speakers or other mass gatherings. For Cohort #1, graduation took 
place on Zoom (though some RAM staff organized a noisy caravan of decorated cars that 
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travelled to the homes of RAM grads). Even more disruptions were created for Cohorts #2, #3, 
and #4. 


For example, the summer REUs for which RAM Scholars has been accepted were all cancelled. It 
should be mentioned, however, that RAM-affiliated faculty in various disciplines did step up 
and created a new type of remote research project opportunity over this past summer. As is 
indicated in this paper’s evaluation section, some data collection — particularly in the qualitative 
realm — was limited by the pandemic. As a consequence, some of the measures of student 
perceptions and attitudes regarding their college experiences are missing or constrained by 
smaller than needed responses. In the current situation, seeking responses from the members 
of the control group is more difficult. FSC must await the return to normalcy before those 
results can be obtained from future cohorts and properly analyzed. 


One final lesson should be shared. While many programs disappear after their grant money has 
been exhausted, the FSC RAM Program has a bright future before it. With rigorous evaluation, 
measurable results, and a wide circle of supporters (including the College President and 
Provost), the College has picked up the key expenses that were originally underwritten by the 
grant. Support will only grow as RAM Scholars become RAM alumni, the skilled local workforce 
and local economy profit from the presence of those alumni, and both the parents of RAM 
students and local legislators come to understand the savings realized when students progress 
to graduation in just four years. 
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Appendix 1 
Table 1. Contrasting FSC and UCLA Characteristics of Full Time Undergraduates — IPEDS data Fall 2018 
Total SAT 
Total I fetes a peas SAT | act | Yeari-2 mead aed 
| Ing . « | Retention | Graduation | Graduation 
Enrollment | students Writing* Math Comp & 4 
FSC 7,575 1,315 580 580 24 84% 31% 53% 
UCLA 31,000 6,233 740 780 34 97% 74% 91% 


*75" Percentile 


*2010 Cohort 
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Appendix 2 


Table 2. Overview of Progression by Cohort and Study Condition Assignment 


(28%) (10%) 


C1 C2 C3 C4 
RAM _ Control RAM _ Control RAM _ Control RAM Control 
Randomly Assigned 206 156 205 165 232 151 208 168 
Confirmed Eligibility ° 206 155 204 165 232 151 197 152 
Completed vid 191 144 193 152 217 137 183 136 
P (93%) (93%) (95%) (92%) (94%) (91%)| (93%) (89%) 
163 118 162 119 191 115 
b 
Completed (87%) (83%) (84%) (78%); (88%) ~~: (84%) nia nya 
146 109 150 100 
b 
Completed Yr3 (90%) (92%) (93%) (84%) n/a n/a n/a n/a 
142 95 
b 
Completed Yr4 (97%) (88%) n/a n/a n/a n/a n/a n/a 
101 61 
Cc 
Graduated (49%) (39%) n/a n/a n/a n/a n/a n/a 
; 136 163 
RAMily Scores (66%) 0 (--) (80%) 0 (--) n/a n/a n/a n/a 
ath ote te 195 1 200 231 195 
Participation Scores (95%) (<1%) (98%) 0 (--) (100%) 0 (--) (04%) 0 (--) 
peacttie 119 73 155 30 158 27 172 1 
y (58%) (47%) (76%) (18%) (68%) (18%)| (87%) (<1%) 
Post-survey 4 - ° n/a n/a n/a n/a n/a n/a 


* Matriculation was a criteria for eligibility, but random assignments had to be made in summer to 


accommodate registration and course planning. This is the figure used for calculating attrition. 


> Percentages continuing each year are based on the total from the prior year 


© Percentage graduated is based on entering cohort. 


4 Percentage post-survey response is based on the number of graduating students invited. 
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Appendix 3 


Table 3. Random Assignment by Number of Interventions Experienced 
Randomly Assigned Condition 


Control Treatment Total 
0 622 99.8% 19 2.3% 641 43.8% 
1 0 0.0% 5 0.6% 5 0.3% 
Number of 2 0 0.0% 12 1.4% 12 0.8% 
Interventions 3 0 0.0% 173 20.6% 173 11.8% 
Experienced 4 1 0.2% 330 39.3% 331 22.6% 
5 0 0.0% 238 28.4% 238 16.3% 
6 0 0.0% 62 7.4% 62 4.2% 
Total 623 100.0% 839 100.0% 1,462 100.0% 
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Appendix 4 
Table 4. Rates of Participation by Intervention for RAM Students by Cohort 
C1 C2 C3 C4 

RAM N=206 N=204 N=232 N=197 
FYE 195 94.7% 197 96.6% 224 96.6% 188 95.4% 
IAC 195 94.7% 200 98.0% 232 100.0% 197 100.0% 
RMP 195 94.7% 200 98.0% 224 96.6% 188 95.4% 
CLW 86 41.7% 65 31.9% 93 40.1% 83 42.1% 
SYE 156 75.7% 157 77.0% 184 79.3% 4 2.0% 
MRE 80 38.8% 58 28.4% 29 12.5% 14 7.1% 


FYE: First Year Experience course 
IAC: Intensive Academic Counseling 


RMP: Digital Roadmap to 4-year graduation 


CLW: Collaborative Learning Workshop 
SYE: Second Year Experience course (introduction to research) 
MRE: Mentored Research Experience 
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Appendix 5 


Table 5. Standardized Beta Coefficients for "Dosage" by Outcome Measure 


N B sig. 
GPA Y12 1,353 .084 .001 
Credits Earned Y1 1,353 .232 .000 
On Track Y1 © 1,353 1.211 .000 
Returned Y2 1,462 .115 .000 
GPA Y2°¢ 868 .080 .012 
Credits Earned Y2 868 .166 .000 
On Track Y2 868 1.193 .000 
Returned Y3 1,113 .092 .002 
GPA Y3 ¢ 505 .137 .001 
Credits Earned Y3 505 .149 .001 
On Track Y3 505 1.196 .000 
Returned Y4 ¢ 724 .214 .000 
GPA Y4‘ 237 .047 .462 
Credits Earned Y4 237 .002 .973 
On Track Y4 237 1.119 .041 
Cumulative GPA & 1,462 .129 .000 
Graduated 1,462 1.138 .001 


* Year 1 outcomes available for C1, C2, C3, and C4 

> Beta for “On Track” logistic regressions is the exponentiation of the B coefficient, which is an odds ratio 
© Year 2 outcomes available for C1, C2, and C3 

4 Year 3 outcomes available for C1 and C2 

© Six cohort 1 students graduated prior to their fourth year; total N is remaining C1 and C2 students 

f Year 4 outcomes available for C1 only 

® Cumulative GPA includes all available years for each cohort 

h Graduated includes all cohorts 
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Appendix 6 
Table 6. Bivariate Correlations between Scores for Dosage, RAMily, and Key Outcomes 
N Dosage RAMily Graduated Cum GPA 

RAMily 1462 0,47 ** -- 
Graduated 1462 0,11 ** 0,31 ** -- 
CumGPA 1361 0,08 ** 0,12 ** 0,12 ** -- 
OnTrackY1 1353 0,20 ** 0,13 ** 0,08 ** 0,43 ** 
OnTrackY2 868 0,19 ** 0,15 ** 0,15 ** 0,48 ** 
OnTrackY3 505 0,20 ** 0,23 ** 0,25 ** 0,41 ** 
OnTrackY4 237 0,12 0,21 ** 0,49 ** 0,30 ** 


** Correlation is significant at the 0.01 level (2-tailed). 


FSC RAM Program Outcomes 
Appendix 7: RAM Student Voices: Reflections on Research 


Student #1 
Major: Applied Mathematics 


Experience Type: Internship with the Internal Revenue Service (IRS) 
Location: IRS, Washington, DC 


Project Focus: Predict the amount of enforced revenue that will be collected in the next fiscal 
year 


The Experience: “| was added to a team with four other data scientists. The goal of my project 
was to use different forecasting models to predict how much enforced revenue the IRS will be 
receiving in the near future. | used SQL to pull in data from the IRS databases and used R to set 
up the data, visualize relationships between different variables using graphs, and analyze 
trends. We also used econometric models to predict the state of the economy as it has a strong 
relationship with tax revenue. My work was left with my former co-workers and will be used to 
settle taxing decisions.” 


The Benefit: “The first few weeks were the most stressful as | was essentially crafting a 
capstone project in a field | have never touched before while utilizing software | was 
inexperienced with. With enough perseverance, | was able to complete a project that has 
greatly affected the federal government all while receiving training for essential skills on the 
job. Solving problems that have never been tackled before always brought me a feeling of 
accomplishment which was essential to building my motivation. Living away in Washington D.C. 
for three months not only taught me professional skills, but also granted me the potential to 
strive further in my career and realize that it's best to live life to the fullest.” 


Words of Wisdom: “I would never have thought | would be an intern working a major project 
that would benefit the IRS. Competing for an extremely selective position and succeeding made 
me realize that anything is possible. It's best to start early and start searching for opportunities 
as early as possible. As Rabindranath Tagore once said, ‘you can't cross the sea merely by 
standing and staring at the water.’ 


Student #2 
Major: Bioscience 


Experience Type: Off-campus research experience 
Research Mentor: Rosalia Davi 


Location: Stony Brook University 
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Project Focus: The Effects of Low Intensity Vibrations on the Cytoskeleton of T Cells for CAR T 
Cancer Therapy 


The Experience: “During my summer, | was enrolled into the REU at Stony Brook University for 
Nanotechnology, Health and Environment. | was able to work with Dr. Mei Lin Chan in the 
Department of Biomedical Engineering. | worked on a new cancer therapy called Chimeric 
Antigen Receptor T Cell therapy. CAR T cell therapy is a new form of immunotherapy that 
specially alters T cells to target cancer cells. | presented my research to experts in the field. This 
research is important because it can have a large impact on the medical field as well as a 
deeper understanding of CAR T therapy.” 


The Benefit: “This research experience benefited me because it allowed me to gain a deeper 
understanding of what it’s like working in a prestigious research facility. | was able to attend 
weekly seminars on how to conduct research and the potential careers in the field. This 
experience has allowed me to grow as an individual and it has made me very motivated to see 
what impact my research could have on the science community as well as the medical field. 
This experience made me develop a deeper passion to work in the field of medicine to learn 
more about different illnesses and to help patients.” 


Words of Wisdom: “Doing research can open up many opportunities for the future. It also very 
interesting getting to focus on a project and learning the different techniques required to 
complete it. Choose a subject that interests you so that you enjoy working on it.” 


Project Funding Source: National Science Foundation, Research Experiences for 
Undergraduates 


Student #3 
Major: Applied Psychology 


Experience Type: On-campus research experience in personality perception on social media 
Research Mentor: Dr. Sayeed Islam (FSC) 

Location: Farmingdale State College 

Project Focus: Perceptions of personality through Instagram profiles 


The Experience: “| worked with Dr. Islam on a research idea | proposed in my RAM 201 course. | 
was responsible for reading many related research articles, planning out how | want to conduct 
my research, what profiles | want to focus on, creating a survey using Qualtrics, and getting IRB 
approval to conduct my research. | have already had the opportunity to present my research 
proposal at the SUNY Applied Learning Conference in Tarrytown, NY. | have future 
presentations planned to disseminate my research.” 
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The Benefit: “| have learned to appreciate research, understand that research is not scary, and 
that research is another format for learning. Research is something that takes a lot time and 
commitment. | know that my involvement in conducting my own research with the guidance of 
Dr. Islam will most definitely help me get into graduate school because not many students have 
the opportunity to say ‘Yes, | conducted my own research while acquiring my undergraduate 
degree.’” 


Words of Wisdom: “Research seems scary and long. However, if you break up the research into 
manageable chucks with a time frame you can work with, then you can do it. A big plus side is 
when you find a topic that interests you, research will become intriguing because you will 
constantly discover new things along the entire journey.” 


*This research study has been published in the Journal of Management and Innovation with Dr. 
Sayeed Islam as co-author. 


Student #4 
Major: Business Management 


Experience Type: On-campus research experience in blended interactions 
Research Mentor: Dr. Brian O’ Keefe (FSC) 

Location: Farmingdale State College, New York, NY, & Patchogue, NY 
Project Focus: To enhance user experiences using mixed reality 


The Experience: “| worked on a team of students from Farmingdale State College and 
Edinburgh-Napier University in Scotland to develop mixed reality prototypes. | was responsible 
for giving input and developing ideas. We visited NYC’s High Line Park and developed a virtual 
tour of the park. We also created several mixed reality proposals that would occupy the 
landscape. With another student, | designed a mixed reality game app that would help people 
visiting the High Line be more environmentally conscious about the Hudson River. We 
presented our work at Mufieca Arthouse in Patchogue, NY. This work will continue in Scotland 
in summer 2019.” 


The Benefit: “| was paid for this experience while | made new connections and gained valuable 
knowledge. | was able to meet new people. | also learned about new and emerging 
technologies and practice my public speaking skills.” 


Words of Wisdom: “RAM research is a great way to expand your mind. It allows you to meet 
new people that have similar goals as you. The RAM Program constantly opens new doors.” 
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Student #5 
Major: Computer Programming & Information Systems 


Experience Type: Internship in web development 
Location: Nature’s Bounty Co., Ronkonkoma, NY 


Project Focus: Full-stack development of business branded website and a business analytics 
project 


The Experience: “For the website, | worked alongside a mentor and was responsible for the 
design, coding, mobile implementation, search engine optimization and overall efficiency of the 
website. The website is now live at spoonfuls.solgar.com. For the business analytics project, | 
worked with 6 other interns to analyze a critical business problem and propose a solution to it 
while also testing our solution to the entirety of the domestic consumer base. We also gave a 
presentation to the C-level executives (CEO, CIO, etc.) on the work we did and how 
implementing our plan could benefit the company.” 


The Benefit: “This experience bolstered my ‘soft skills’ such as teamwork and communication 
and honed my technical knowledge in web development (both front-end and back-end). It also 
allowed me the opportunity to work with powerful business analytics tools and programming 
technologies | may not have had the chance to work with otherwise. This internship made me 
develop a deeper passion for web development and programming while also working with a 
knowledgeable and caring team.” 


Words of Wisdom: “To those seeking to complete their RAM research experience, | would 
recommend looking beyond your major into similar fields since you never know what could 
trigger a response in you that makes clearer or even changes your career goals. For internships 
specifically, give your all when you work and learn as much as you can since your time there 
goes by so quick and you can learn valuable skills if you open up to those around you, even if 
the topic has nothing to do with your current projects (employers and your colleagues will take 
note of your initiative to learn). | recommend that when you are given work, be your own critic. 
Go over every little detail that can be done better or made faster. Your work matters here! 
Show your manager or boss that you are meticulous, put your best foot forward, and don't be 
afraid to ask, ‘How can | do this better?” 


This last point | want to say without putting pressure on you: Remember that whether you are 
doing research or an internship, you represent not only yourself, but also the RAM Program and 
Farmingdale State College. Keep that in mind, look for ways to grow yourself and hold yourself 
accountable for whatever you do. While you shouldn't lose sleep over your work, you should be 
thinking of ways to truly showcase what you can do. Go the extra mile in your work or for 
someone else. In whatever you do, show what it means to be a RAM scholar.” 
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Student #6 
Major: Computer Engineering 


Experience Type: Off-/On-campus research experience 
Research Mentor: Dr. Paulo Castillo and Kwang Min Yu 
Location: Farmingdale State College and Brookhaven National Lab, NY 


Project Focus: A numerical study of coherent radiation from an induced plasma dipole 
oscillation created by two detuned lasers. 


The Experience: “| worked with a partner to run simulations from a code created by my mentor 
to then analyze the physical theory and to help develop programs to better analyze the 
simulation data. Our actions helped refine the simulation code. A poster presentation was given 
at the end of the internship. Additionally, a paper using data collected from our simulations is in 
the process of being published. | am continuing this project by running simulations for the 
researcher to analyze.” 


The Benefit: “This experience has provided a deeper insight into coding and particle physics. | 
was able to meet like-minded individuals from universities from across the county. Additionally, 
this internship provided me a graduate level experience that has given me a better 
understanding of how research would be conducted within a graduate school. Overall, this 
experience has helped me gain a new perspective on my current career path.” 


Words of Wisdom: “Always do the best that you can do and don't worry too much about any 
set-backs you may face. Everything you do is something that you can learn from.” 


Project Funding Sources: Brookhaven National Laboratory, United States Department of Energy 
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